{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some features to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# As usual, a bit of setup\n",
    "\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_val:  (1000, 3, 32, 32)\n",
      "X_train:  (49000, 3, 32, 32)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "y_train:  (49000,)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.iteritems():\n",
    "  print '%s: ' % k, v.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.3\n",
      "Mean of input:  10.0000955992\n",
      "Mean of train-time output:  10.024859874\n",
      "Mean of test-time output:  10.0000955992\n",
      "Fraction of train-time output set to zero:  0.699228\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.6\n",
      "Mean of input:  10.0000955992\n",
      "Mean of train-time output:  9.99359713673\n",
      "Mean of test-time output:  10.0000955992\n",
      "Fraction of train-time output set to zero:  0.400364\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.75\n",
      "Mean of input:  10.0000955992\n",
      "Mean of train-time output:  9.98423976413\n",
      "Mean of test-time output:  10.0000955992\n",
      "Fraction of train-time output set to zero:  0.2511\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.3, 0.6, 0.75]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print 'Running tests with p = ', p\n",
    "  print 'Mean of input: ', x.mean()\n",
    "  print 'Mean of train-time output: ', out.mean()\n",
    "  print 'Mean of test-time output: ', out_test.mean()\n",
    "  print 'Fraction of train-time output set to zero: ', (out == 0).mean()\n",
    "  print 'Fraction of test-time output set to zero: ', (out_test == 0).mean()\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  5.44560773382e-11\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.8, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "print 'dx relative error: ', rel_error(dx, dx_num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specificially, if the constructor the the net receives a nonzero value for the `dropout` parameter, then the net should add dropout immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  0\n",
      "Initial loss:  2.30145865756\n",
      "W1 relative error: 7.11e-07\n",
      "W2 relative error: 1.17e-06\n",
      "W3 relative error: 3.42e-08\n",
      "b1 relative error: 7.11e-08\n",
      "b2 relative error: 4.09e-09\n",
      "b3 relative error: 8.76e-11\n",
      "\n",
      "Running check with dropout =  0.25\n",
      "Initial loss:  2.30184686688\n",
      "W1 relative error: 3.11e-07\n",
      "W2 relative error: 3.07e-06\n",
      "W3 relative error: 8.70e-08\n",
      "b1 relative error: 8.90e-09\n",
      "b2 relative error: 2.26e-09\n",
      "b3 relative error: 1.00e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.30606121151\n",
      "W1 relative error: 4.68e-07\n",
      "W2 relative error: 2.75e-07\n",
      "W3 relative error: 1.92e-07\n",
      "b1 relative error: 2.55e-08\n",
      "b2 relative error: 1.25e-09\n",
      "b3 relative error: 1.31e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [0, 0.25, 0.5]:\n",
    "  print 'Running check with dropout = ', dropout\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print 'Initial loss: ', loss\n",
    "\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print '%s relative error: %.2e' % (name, rel_error(grad_num, grads[name]))\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a dropout probability of 0.75. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "(Iteration 1 / 125) loss: 8.138254\n",
      "(Epoch 0 / 25) train acc: 0.180000; val_acc: 0.159000\n",
      "(Epoch 1 / 25) train acc: 0.406000; val_acc: 0.242000\n",
      "(Epoch 2 / 25) train acc: 0.418000; val_acc: 0.235000\n",
      "(Epoch 3 / 25) train acc: 0.586000; val_acc: 0.240000\n",
      "(Epoch 4 / 25) train acc: 0.706000; val_acc: 0.295000\n",
      "(Epoch 5 / 25) train acc: 0.742000; val_acc: 0.266000\n",
      "(Epoch 6 / 25) train acc: 0.760000; val_acc: 0.272000\n",
      "(Epoch 7 / 25) train acc: 0.830000; val_acc: 0.266000\n",
      "(Epoch 8 / 25) train acc: 0.882000; val_acc: 0.280000\n",
      "(Epoch 9 / 25) train acc: 0.910000; val_acc: 0.292000\n",
      "(Epoch 10 / 25) train acc: 0.954000; val_acc: 0.296000\n",
      "(Epoch 11 / 25) train acc: 0.966000; val_acc: 0.282000\n",
      "(Epoch 12 / 25) train acc: 0.962000; val_acc: 0.265000\n",
      "(Epoch 13 / 25) train acc: 0.980000; val_acc: 0.288000\n",
      "(Epoch 14 / 25) train acc: 0.980000; val_acc: 0.305000\n",
      "(Epoch 15 / 25) train acc: 0.992000; val_acc: 0.295000\n",
      "(Epoch 16 / 25) train acc: 0.988000; val_acc: 0.287000\n",
      "(Epoch 17 / 25) train acc: 0.982000; val_acc: 0.280000\n",
      "(Epoch 18 / 25) train acc: 0.996000; val_acc: 0.275000\n",
      "(Epoch 19 / 25) train acc: 0.984000; val_acc: 0.273000\n",
      "(Epoch 20 / 25) train acc: 0.994000; val_acc: 0.288000\n",
      "(Iteration 101 / 125) loss: 0.015326\n",
      "(Epoch 21 / 25) train acc: 0.998000; val_acc: 0.290000\n",
      "(Epoch 22 / 25) train acc: 0.992000; val_acc: 0.304000\n",
      "(Epoch 23 / 25) train acc: 0.998000; val_acc: 0.296000\n",
      "(Epoch 24 / 25) train acc: 0.998000; val_acc: 0.292000\n",
      "(Epoch 25 / 25) train acc: 0.998000; val_acc: 0.289000\n",
      "0.75\n",
      "(Iteration 1 / 125) loss: 8.321506\n",
      "(Epoch 0 / 25) train acc: 0.220000; val_acc: 0.177000\n",
      "(Epoch 1 / 25) train acc: 0.380000; val_acc: 0.252000\n",
      "(Epoch 2 / 25) train acc: 0.454000; val_acc: 0.246000\n",
      "(Epoch 3 / 25) train acc: 0.590000; val_acc: 0.277000\n",
      "(Epoch 4 / 25) train acc: 0.668000; val_acc: 0.274000\n",
      "(Epoch 5 / 25) train acc: 0.772000; val_acc: 0.283000\n",
      "(Epoch 6 / 25) train acc: 0.794000; val_acc: 0.297000\n",
      "(Epoch 7 / 25) train acc: 0.820000; val_acc: 0.282000\n",
      "(Epoch 8 / 25) train acc: 0.884000; val_acc: 0.262000\n",
      "(Epoch 9 / 25) train acc: 0.928000; val_acc: 0.279000\n",
      "(Epoch 10 / 25) train acc: 0.914000; val_acc: 0.292000\n",
      "(Epoch 11 / 25) train acc: 0.950000; val_acc: 0.286000\n",
      "(Epoch 12 / 25) train acc: 0.934000; val_acc: 0.261000\n",
      "(Epoch 13 / 25) train acc: 0.930000; val_acc: 0.284000\n",
      "(Epoch 14 / 25) train acc: 0.964000; val_acc: 0.310000\n",
      "(Epoch 15 / 25) train acc: 0.976000; val_acc: 0.310000\n",
      "(Epoch 16 / 25) train acc: 0.980000; val_acc: 0.300000\n",
      "(Epoch 17 / 25) train acc: 0.972000; val_acc: 0.296000\n",
      "(Epoch 18 / 25) train acc: 0.988000; val_acc: 0.319000\n",
      "(Epoch 19 / 25) train acc: 0.998000; val_acc: 0.314000\n",
      "(Epoch 20 / 25) train acc: 0.998000; val_acc: 0.311000\n",
      "(Iteration 101 / 125) loss: 0.002297\n",
      "(Epoch 21 / 25) train acc: 0.998000; val_acc: 0.324000\n",
      "(Epoch 22 / 25) train acc: 0.998000; val_acc: 0.317000\n",
      "(Epoch 23 / 25) train acc: 0.996000; val_acc: 0.310000\n",
      "(Epoch 24 / 25) train acc: 0.998000; val_acc: 0.315000\n",
      "(Epoch 25 / 25) train acc: 0.998000; val_acc: 0.319000\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [0, 0.75]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print dropout\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4AAAAJXCAYAAADCeDp0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xt8XWWZ6PHf0xYSoKVUQC4V6BhRAUdEpeKAEC+QYlS8\nDrYKgjowM5LUmfGo3MbMQY6Dc7MNHIUBbziAHAVFo6V1hlAVVJCCOhQHwp0CCpU7Cb0854+9G3fS\n3ZI0yV473b/v55NP13rXu9Z6srO6s5+8t8hMJEmSJElbvylFByBJkiRJqg0TQEmSJElqECaAkiRJ\nktQgTAAlSZIkqUGYAEqSJElSgzABlCRJkqQGYQIoSdpqRcQPIuK4ouOQJKlehOsASpLqSUQ8BWz4\n5bQD0A+sK++flJmXFhKYJElbARNASVLdioi7gI9k5n9VOTYtM9cWENaEi4gASH9JS5LGmV1AJUmT\nQkS0RsT9EfHJiHgQuCgidoqI70fE7yJidUR8LyJmV5zTGxEfKW+fEBE/iYh/Kte9MyLmbeZ+n46I\nOyLiiYj474h457DjfxERt1YcP6hcvldEXFGO6ZGI6C6Xd0XExRXnz4mI9RExpSLWz0bET4GngRdH\nxIkV9+iLiJOGxXBMRNwcEY+XY22LiPdFxI3D6v1tRHxnC196SdJWxARQkjSZ7AbMAvYGTqb0e+yi\n8v7ewLPAuRX1kz92JwWYC9wG7Ax8vnzuptwBHJaZOwL/AHwjInYDiIj3AZ8BjisffwfwaERMBb4P\n3AXsA8wGNnRZHUlr3geBjwLTgXuAh4H28j1OBP6tItGcC3wN+LvMnAkcDtwNfBf4k4h4ecV1jyvX\nlSQ1OBNASdJksh74TGauycz+zFydmVeWt58C/g9wxGbOvyczLyp3rfw6sEdEvLBaxcz8VmY+VN6+\nHLidUgIJpSTtnMz8Zfl4X2beWz6+B/C/MvPZzBzIzOvK58TzfG8JfDUzV2bm+sxcm5k/yMy7yvdY\nDiwF3lCu/xHgosz8z/LxVZn528x8DvgmpWSSiDiAUjL6/ee5vySpAZgASpImk9+XExwAImL7iDg/\nIu6OiMeBa4GZG8bQVfHQho3MfKa8Ob1axYg4PiJWRMQfIuIPwCuAXcqHXwT0VTltL0pJ5vrRfVuD\n7hsWw9ER8bOIeLQcw1sptV5uLgYotfYtKG8fB3wzM9dsYUySpK2ICaAkaTIZ3o3y74CXAnPL3SCP\noNTS9nytbZsVEfsAFwAfA16QmbOA31Rc9z7gJVVOvQ/Yu9wVdLingO0r9nevUmfw+4uIJuDblLqq\nvrAcww9GEAOZ+XPguYg4HJgPXFytniSp8ZgASpIms+mUxv09HhEvoDQubzzsQCkZewSYEhEnUmoB\n3OBC4BMR8eooeUlE7A38HHgQ+Mdy62RzRPxZ+ZybgcPLk8TMBE6tct/KxHXb8tcjwPqIOBo4quL4\nRcCJEfGmiJgSEbMj4mUVx79OaTzkcxXdUCVJDc4EUJI0mQxvAfwCsB2lJOk64IdV6lSeO/xY1bqZ\neSvwL8D1lLqNvgL4ScXxbwFnA5cATwBXALPKXT/fTqll7l5KrXR/Xj7nR5TG5v0KuAH43ubiycwn\ngU7gcmA1pZa871Ycv4HyxDDAY8A1lCbC2eBi4ADgG5t4PSRJDaiQdQAj4stAO/C7zPzTTdRZDBwN\nPAOckJkrahiiJEmTWkRsR2kW0YMyc1NjBSVJDaaoFsCvAJtbe+mtwEsyc1/gJOCLtQpMkqStxF8B\nvzD5kyRVmlbETTPzxxExZzNV3kF5vaLM/Hl5od/dMvPhWsQnSdJkFhF3U+pO+s7nqSpJajCFJIAj\nMJuhU2HfT2m6axNASZKeR2bOKToGSVJ9qtcEEDaewnujwYoRUfsBjJIkSZJURzJzxMsf1WsC+ACl\nxXQ3eFG5bCNFTGIjjURXVxddXV1FhyFtxGdT9aSt7QyWLv1sea+r/AVtbWeyZMlZNYujp2c5Cxde\nTV/f2YNlLS2ns2hRG+3th9csjnrwitcs4L8HfgHvqxg++v9aOKBpLr/55SU1iaH1hFau/ZNrNyo/\n4q4j6P1qb01iAJj16n147Jh7SzvXAG8sl393b1bfdE9NYtjpVXN4/F0b32vmlfvw2M131yQGGPZa\nVJY34GtRD3EMiaFrdOfW6zIQVwHHA0TEIcBjjv+TJGnrMzBQ/W/R/f1TaxrH4sVLhyR/AH19Z9Pd\nvaymcfT0LKet7QxaW7toazuDnp7lNb0/wENrfjY0+QN4Xx8Prf15zWJoiqaq5c1TmmsWA0A8t4m2\nkue2qVkM09ZX/7g+LWv7f8TXor7i2FQMI1FIAhgRl1Jar+llEXFfRHw4Ik6OiJMBMvMHwJ0RcQdw\nPvDXRcQpSdJEqYcP+vUQR1PT2qrlzc3rahrHwMA02LYH9myDfVpL/27bU9NEdEMr5NKln+Xaa7tY\nuvSzLFx4dc1/Jnvss2P18r1n1CyG1+/7JqZdudOQsmlX7sQhL3ljzWIAmDP9tfD/WoYWXt7CnBmv\nrVkMp7z/5KqvxSnHnlSzGMDXot7iqBbDSBU1C+j8EdQ5pRaxSBOltbW16BCkqnw2i1etu2Ff3+kA\nNe1uWA9xdHYexa9+++c8tOZxWPcwTL2e3bfZkY6Ojprcf4Mn+lfCvt8Y1u2xjycGavfhdtOtkGfW\n9LnYc5fd+E2V8tm77l6zGK6/9knWrvwG/L4btumHNc2sfaSDny3/GZxWszA468yP8dGPnctDF7wE\n1j0Et+/O7tvM4KzP165touu0TwFw7jcvYG2sY1pO5ZRjTxosr5Uhr0X5Z9Kor0U9xFEZw6PcOapz\nC1kIfrxERE7m+CVJjWnouLfK8tqOe6uHOHqW9fDRz5/EQ4etGizb/Sd7cuEnL6D9yPaaxADw6rfN\nZcXBN2xcfuNcfvm92nR9bG3t4trrD4ZdFsM2A7CmCR7p5IjX30Bvb1dNYoDSz2TheQvpO+iPyXDL\nTS0sOmVRzX4mra1dXHtt10blRxzRVdPXAkp/KOnuXkZ//1Sam9fR0XFkw40L3cDXoj5FxFYxCYwk\nSVutehn3Vg9xLL5k8ZDkD+Chw1bRfWl3TRPAHXfZvmr5jJ23q1kM9dAKCQy+7t2XdtO/vp/mKc10\nnNJR059HvXQNhlJruElOia/F1sEEUJJUMz09y1m8eCkDA9NoalpLZ+dRDflhol4+3DY1rS2NexvW\n4lTLOAZyoGp5//r+msUAdTLpyC53wdEbT74SN+5cuxjK2o9sr2nCN1xn51H09Z0+bFbW0+jomFdY\nTNLWwgRQklQT9TDerDKWIhPRevlw+/ojZvBfD3+Qte96bLBs2pW/4JDDP12zGOoi8QI6F3TSd17f\nRt0eO06p3VjEemiFrBcb/j92d59Z0d1wXkP+wUgabyaAkqSaqJcJLuohEa2XD7fX3/5fQ5I/gLXv\neoyf3XENUJvJDOoh8YI66fZYJ8lwvbC7oTQxTAAlqUEU3epVD+PNoH4S0Xr4cFsP3S/rIfGqjKXQ\nbo91kgxL2rqZAEpSA6iHVq96GG8G9ZOI1oN6aXEqOvGqF/WUDEvaepkASlIDqIdWr3oYbwb1MwFL\nPbDFqf6YDEuaaCaAktQA6qHVqx7Gm0H9TMBSD2xxkqTGYwIoSQ2gHlq96mG8GdTPBCz1whYnSWos\nU4oOQJI08To7j6Kl5fQhZaVWryNrFkO9jDcDYNsnyT1+AXN6S/9u+2TtY5AkqQC2AEpSA6iHVq96\nGW/Ws6yHhectHBJH33ml7Vq2hPUs62HxJYsZyAGaoonOBZ22xEmSJlxkZtExbLGIyMkcv6TG4Yf9\nkp5lPUPHm82v/XizthPbWDpn6cbl97Sx5MtLahJDtSS0ZUULiz62qCGfC0nSlosIMjNGWt8WQEma\nYPXS4lQP6mG8WT2MRVx8yeIhzwNA30F9dF/aXfjrI0naujkGUJIm2OY+7Kv26mEsYj0koZKkxmQC\nKEkTzA/79aVzQSctK1qGlLXc1ELH/NqNRayHJFSS1JjsAipJE6xePuw7DrGkHta+q5cJcSRJjccE\nUJImWOeCTn71+d/w0GGrBst2/8medHyydh/2HYc4VNFjEeshCZUkNSYTQEmaaM/NgNsPhVufgG36\nYU0zbLNjqbxGnHSk/hSdhEqSGlNhYwAjYl5E3BYRt0fEp6ocnxURV0bELRHx84g4oIg4JWmsFi9e\nykP3XA6rlsA9vbBqCQ/dcznd3ctqFoPjECVJEhSUAEbEVOBcYB6wPzA/IvYbVu004KbMPBA4HlhU\n2yglaXwMDFTvbNHfP7VmMdTLOERJklSsoloA5wJ3ZObdmbkGuAw4Zlid/YBrADLzt8CciNi1tmFK\n0tg1Na2tWt7cvK5mMdTDzJeSJKl4RY0BnA3cV7F/P/C6YXVuAd4N/CQi5gL7AC8Cfl+TCCVpnHR2\nHkVf3+n09Z09WNbSchodHfNqFoOTjkiSJCguAcwR1PlHYFFErAB+DawANvpzeVdX1+B2a2srra2t\n4xOhJI2T9vbDAejuPpP+/qk0N6+jo2PeYHnN4nDSEUmSJr3e3l56e3u3+PzIHEkuNr4i4hCgKzPn\nlfdPBdZn5jmbOecu4E8z86mKsiwifkmTR9f/OYdzLzuftVPWM239FE55/8l0nbbRvFOSJEmTUkSQ\nmTHS+kW1AN4I7BsRc4BVwLHA/MoKETETeDYzn4uIvwCurUz+JOn5dP2fczj78n9k7XseGyw7+/J/\nLB0zCZQkSQ2okBZAgIg4GvgCMBW4KDM/FxEnA2Tm+RHxeuCrlLqL/gb4SGY+PuwatgBK2qRdXvli\nHn3PXRuV73zFi3nklr4qZ0iSJE0uk6UFkMz8IfDDYWXnV2xfD7ys1nFJ2nqsnbK+ennUbvZNSZKk\nelLYQvCSNNGmra/+Fjcta7f+niRJUj0xAZS01Trl/Scz7cqdhpRNu3InTjn2pIIikiRJKlZhXUAl\naaJtmOjl3G9ewNpYx7ScyinHnuQEMJIkqWEVNgnMeHASGEmSJEmNbLSTwNgFVJIkSZIahAmgJEmS\nJDUIE0BJkiRJahAmgJIkSZLUIEwAJUmSJKlBmABKkiRJUoMwAZQkSZKkBuFC8JImTE/PchYvXsrA\nwDSamtbS2XkU7e2HFx2WJElSwzIBlDQhenqWs3Dh1fT1nT1Y1td3OoBJoCRJUkHsAippQixevJS+\n+/4M9myDfVphzzb67vszuruXFR2aJElSw7IFUNKEeOCRO2Hfy+B9fX8s/H993P/7ucUFJUmS1OBs\nAZQ0IR5a87OhyR/A+/p4aO3PiwlIkiRJJoCSJsYe++xYvXzvGTWORJIkSRuYAEqaEHvuslvV8tm7\n7l7jSCRJkrRBYQlgRMyLiNsi4vaI+FSV4zMj4nsRcXNE/CYiTiggTElbqHNBJy0rWoaUtdzUQsf8\njoIikiRJUmRm7W8aMRX4LfAW4AHgBmB+Zq6sqHMaMCMzT42IXcr1d8vMtRV1soj4JY1Mz7Ieui/t\npn99P81TmumY30H7ke1FhyVJkrTViAgyM0Zav6hZQOcCd2Tm3QARcRlwDLCyos56YMMgoh2BRyuT\nP0n1r/3IdhM+SZKkOlJUAjgbuK9i/37gdcPqnAt8LyJWATOAP69RbNKk17Osh8WXLGYgB2iKJjoX\ndJqISZIkqbAEcCT9NucBN2XmGyOiBVgWEQdm5pOVlbq6uga3W1tbaW1tHc84pUmnZ1kPC89bSN9B\nf1yCoe+80rZJoCRJ0uTW29tLb2/vFp9f1BjAQ4CuzJxX3j8VWJ+Z51TU+T7wucz8aXn/P4FPZeaN\nFXUcAygN03ZiG0vnLN24/J42lnx5SQERSZIkaaKMdgxgUbOA3gjsGxFzImJb4FjgqmF17qU0SQwR\nsRvwMuDOmkYpTUIDOVC1vH99f40jkSRJUr0ppAtoZq6NiFOAq4GpwEWZuTIiTi4fPx84C/hqRPwK\nCOCTmbm6iHilyaQpmqqWN09prnEkkiRJqjeFdAEdL3YBlTZWbQxgy00tLDplkWMAJUmStjKj7QJq\nAiiNs3qYgdP19yRJkhqDCaBUoKqtbytaWPQxW98kSZI0/kwApQI5A6ckSZJqabQJYFHrAEpbpXqZ\ngbOnZzmLFy9lYGAaTU1r6ew8ivb2w2sagyRJkuqPCaA0juphBs6enuUsXHg1fX1nD5b19Z0OYBIo\nSZLU4IpaB1DaKnUu6KRlRcuQspabWuiY31GzGBYvXjok+QPo6zub7u5lNYtBkiRJ9ckWQGkcbZjo\nZcgMnKfUdgbOgYHq/637+6fWLAZJkiTVJxNAaZy1H9le6IyfTU1rq5Y3N6+rcSSSJEmqN3YBlcZZ\nT89y2trOoLW1i7a2M+jpWV7T+3d2HkVLy+lDylpaTqOj48iaxiFJkqT6YwugNI7qYQKWDffp7j6T\n/v6pNDevo6NjnhPASJIkyXUApfHU1nYGS5d+tkr5mSxZclYBEUmSJGlrNtp1AO0CKo0jJ2CRJElS\nPTMBlMaRE7BIkiSpnpkASuPICVgkSZJUzxwDKI2znp7ldHcvq5iA5UgnYJEkSdKEGO0YQBNASZIk\nSZqknARGkiRJklSVCaAkSZIkNYjCFoKPiHnAF4CpwIWZec6w458APlDenQbsB+ySmY/VNFBNGj09\ny1m8eCkDA9NoalpLZ+dRjr2TJEmSKhSSAEbEVOBc4C3AA8ANEXFVZq7cUCcz/xn453L9twEfN/nT\npvT0LGfhwqvp6zt7sKyvrzQbp0mgJEmSVFJUF9C5wB2ZeXdmrgEuA47ZTP0FwKU1iUyT0uLFS4ck\nfwB9fWfT3b2soIgkSZKk+lNUAjgbuK9i//5y2UYiYnugDfh2DeLSJDUwUL0xu79/ao0jkSRJkupX\nUWMAR7N2w9uBn2yq+2dXV9fgdmtrK62trWMKTJNTU9PaquXNzetqHIkkSZI0cXp7e+nt7d3i8wtZ\nBzAiDgG6MnNeef9UYP3wiWDKx64EvpmZl1U55jqAAqqPAWxpOY1Fi+Y5BlCSJElbrUmxEHxETAN+\nC7wZWAX8AphfOQlMud5M4E7gRZn5bJXrmABqUE/Pcrq7l9HfP5Xm5nV0dBxp8idJkqSt2qRIAAEi\n4mj+uAzERZn5uYg4GSAzzy/X+RDQlpkLNnENE0BJkiRJDWvSJIDjwQRQkiRJUiMbbQJY1CygkiRJ\nkqQaK2oWUGnc9SzrYfElixnIAZqiic4FnbQf2V50WJIkSVLdMAHUVqFnWQ8Lz1tI30F9g2V955W2\nTQIlSZKkEscAaqvQdmIbS+cs3bj8njaWfHlJARFJkiRJE88xgGpIAzlQtbx/fX+NI5EkSZLqlwmg\ntgpN0VS1vHlKc40jkSRJkuqXCaC2Cp0LOmlZ0TKkrOWmFjrmdxQUkSRJklR/HAOorUbPsh66L+2m\nf30/zVOa6Zjf4QQwkiRJ2qq5ELwkSZIkNQgngZEkSZIkVWUCKEmSJEkNwgRQkiRJkhqECaAkSZIk\nNQgTQEmSJElqECaAkiRJktQgTAAlSZIkqUFscQIYEe+ICBNISZIkSZokxpLAHQvcERGfj4iXj1dA\nkiRJkqSJEZm55SdHzATmAycACXwFuDQznxyX6J7//jmW+CVJkiRpMosIMjNGWn9MXTgz83HgW8A3\ngT2BdwErIqLzeYKcFxG3RcTtEfGpTdRpjYgVEfGbiOgdS5ySJEmSpDG0AEbEMZRa/vYFvg58NTN/\nFxHbA7dm5pxNnDcV+C3wFuAB4AZgfmaurKizE/BToC0z74+IXTLzkSrXsgVQkiRJUsMabQvgtDHc\n693Av2Xm8srCzHwmIj66mfPmAndk5t0AEXEZcAywsqLOAuDbmXl/+ZobJX+SJEmSpNEZSxfQf6DU\negdARGwXEXMAMvNHmzlvNnBfxf795bJK+wIviIhrIuLGiDhuDHGqBnqW9dB2YhutJ7TSdmIbPct6\nig5JkiRJ0jBjaQG8HPiziv31lMYDvvZ5zhtJn81tgFcDbwa2B66PiJ9l5u3DK3Z1dQ1ut7a20tra\nOoLLazz1LOvho58/iYcOWzVY9qvP/4YLuYD2I9sLjEySJEnauvT29tLb27vF549lDODNmfmqYWW3\nZOaBz3PeIUBXZs4r758KrM/McyrqfArYLjO7yvsXAksy81vDruUYwDrw6rfNZcXBN2xcfuNcfvm9\nnxcQkSRJktQYajkL6CPliWA23PgYYCRj9W4E9o2IORGxLaX1BK8aVue7wGERMbU8qczrgFvHEKsm\n0F2rHq5e/sBDNY5EkiRJ0uaMpQvoXwL/ERHnlvfvB553rF5mro2IU4CrganARZm5MiJOLh8/PzNv\ni4glwK8odS3998w0AaxT8dwmHqPntqltIJIkSZI2a0wLwQNExAwgM/Op8QlpVPe2C2gdePUhx7Li\nqV/C+/r+WHh5Cwft+Fpuuv6y4gKTJEmStnKj7QI6pgQwIt4G7A80byjLzP+9xRcc/f1NAOtAT89y\nPvqxc3lozROwTT+saWb3bWZw4XkdtLcfXnR4kiRJ0larZusARsT5wHbAm4B/B94HOONHA2pvP5wL\nge7uZfT3T6W5eR0dHUea/EmSJEl1ZiyzgP46M/80In6Vma+MiOmUZuo8bHxD3GwMtgBKkiRJali1\nnAX02fK/z0TEbGAtsPsYridJkiRJmkBjmQX0exExC/gn4Jflsn8fe0iSJEmSpImwRV1AI2IK8PrM\n/Gl5vxlozszHxjm+54vDLqCSJEmSGlbNZgGNiJsz81VbdPI4MQGUJEmS1MhqOQbwRxHx3ogY8c0k\nSZIkScUZSwvgU8D2wDqgv1ycmbnjOMU2khhsAZQkSZLUsGq2DmBmTt/ScyVJkiRJtTeWheCrrvKd\nmcu3PBxJkiRJ0kQZSxfQ7wMbTm4G5gK/zMw3jVNsI4nBLqCSJEmSGlYtu4C+bdiN9wIWben1JEmS\nJEkTayyzgA53P7DfOF5PkiRJkjSOxjIGsLtidwrwKuCXY45IkiRJkjQhtjgBpJTsbRiAtxa4JDN/\nOvaQJEmSJEkTYSyTwEwHns3MdeX9qUBTZj4zjvE9XwxOAiNJkiSpYY12EpixjAH8EbBdxf725TJJ\nkiRJUh0aSwLYnJlPbdjJzCcpJYEjEhHzIuK2iLg9Ij5V5XhrRDweESvKX2eMIVZJkiRJanhjGQP4\ndES8JjN/CRARrwWeHcmJ5e6i5wJvAR4AboiIqzJz5bCq12bmO8YQoyRJkiSpbCwJ4MeByyPiwfL+\nHsCxIzx3LnBHZt4NEBGXAccAwxPAEfdllSRJkiRt3lgWgr8hIvYDXlYu+m1mPjfC02cD91Xs3w+8\nbvgtgD+LiFsotRJ+IjNv3dJ4JUmSJKnRjWUdwFOA/8jMX5f3Z0XERzPz/47g9JFM3XkTsFdmPhMR\nRwPfAV46vFJXV9fgdmtrK62trSO4tCRJkiRNPr29vfT29m7x+WNZBuKWzDxwWNnNmfmqEZx7CNCV\nmfPK+6cC6zPznM2ccxfwmsxcXVHmMhCSJEmSGlYtl4GYEhGD55cndtlmhOfeCOwbEXMiYltKYwev\nqqwQEbtFRJS351JKVldvfClJkiRJ0kiMZRKYq4HLIuJ8SpO1nAwsGcmJmbm23IX0amAqcFFmroyI\nk8vHzwfeC/xVRKwFngHeP4ZYJUmSJKnhjaUL6FTgJODNlMb0/QrYIzP/evzCe94Y7AIqSZIkqWHV\nrAtoZq4Dfg7cTWlZhzez8TIOkiRJkqQ6MeouoBHxMmA+pS6ZjwLfpNSS2Dq+oUmSJEmSxtOou4BG\nxHrgx8BHM/P2ctldmfknExDf88ViF1BJkiRJDasWXUDfDTwEXBMR/x4Rb6E0CYwkSZIkqY6NZRKY\n6cAxlLqDvhH4OnBlZi4dv/CeNwZbACVJkiQ1rNG2AG5xAjjspi+gtGzD+zPzTWO+4Mjv2/AJYM+y\nHhZfspiBHKApmuhc0En7ke1FhyVJkiSpBkabAI5lHcBB5QXaLyh/qUZ6lvWw8LyF9B3UN1jWd15p\n2yRQkiRJ0nDj0gJYlEZvAWw7sY2lczbucdt2TxtLvrykgIgkSZIk1VLN1gFU8QZyoGp5//r+Gkci\nSZIkaTIwAZzEmqKpannzlOYaRyJJkiRpMjABnMQ6F3TSsqJlSFnLTS10zO8oKCJJkiRJ9cwxgJNc\nz7Ieui/tpn99P81TmumY3+EEMJIkSVKDKGQZiKKYAEqSJElqZE4CI0mSJEmqalzWAVRxenqWs3jx\nUgYGptHUtJbOzqNobz+86LAkSZIk1SETwEmsp2c5CxdeTV/f2YNlfX2nA5gESpIkSdqIXUAnscWL\nlw5J/gD6+s6mu3tZQRFJkiRJqmcmgJPYwED1Btz+/qk1jkSSJEnSZFBIAhgR8yLitoi4PSI+tZl6\nB0fE2oh4dy3jG6menuW0tZ1Ba2sXbW1n0NOzvKb3b2paW7W8uXldTeOQJEmSNDnUfAxgREwFzgXe\nAjwA3BARV2Xmyir1zgGWACOe1rRW6mH8XWfnUfT1nT4khpaW0+jomFeT+0uSJEmaXIqYBGYucEdm\n3g0QEZcBxwArh9XrAL4FHFzT6EZo0+PvzqxZArjhPt3dZ9LfP5Xm5nV0dMxzAhhJkiRJVRWRAM4G\n7qvYvx94XWWFiJhNKSl8E6UEsO5We6+X8Xft7Yeb8EmSJEkakSISwJEkc18APp2ZGRHBZrqAdnV1\nDW63trbS2to61vhGxPF3kiRJkmqtt7eX3t7eLT4/MmvbuBYRhwBdmTmvvH8qsD4zz6mocyd/TPp2\nAZ4B/iIzrxp2rax1/BtUGwPY0nIaixbZBVOSJElSbUQEmTniOVOKSACnAb8F3gysAn4BzB8+CUxF\n/a8A38vMK6ocKywBhFIS2N29rGL83ZEmf5IkSZJqZrQJYM27gGbm2og4BbgamApclJkrI+Lk8vHz\nax3TFtv2SXKPX0AOkNEE2x5SdESSJEmStEk1bwEcT4V2AV3Ww8LzFtJ3UN9gWcuKFhZ9bBHtR7YX\nEpMkSZKkxlL3XUDHU5EJYNuJbSyds3Tj8nvaWPLlJQVEJEmSJKnRjDYBnDKRwWzNBnKgann/+v4a\nRyJJkiTOjhrhAAAgAElEQVRJI2MCuIWaoqlqefOU5hpHIkmSJEkjYwK4hToXdNKyomVIWctNLXTM\n7ygoIkmSJEnaPMcAjkHPsh66L+2mf30/zVOa6Zjf4QQwkiRJkmrGSWAkSZIkqUE4CYwkSZIkqSoT\nQEmSJElqECaAkiRJktQgTAAlSZIkqUGYAEqSJElSgzABlCRJkqQGYQIoSZIkSQ3CBFCSJEmSGoQJ\noCRJkiQ1CBNASZIkSWoQJoCSJEmS1CBMACVJkiSpQZgASpIkSVKDKCwBjIh5EXFbRNweEZ+qcvyY\niLglIlZExA0RcWgRcUqSJEnS1iIys/Y3jZgK/BZ4C/AAcAMwPzNXVtTZITOfLm//KXB5Zu437DpZ\nRPySJEmSVA8igsyMkdYvqgVwLnBHZt6dmWuAy4BjKitsSP7KpgPraxifJEmSJG11ikoAZwP3Vezf\nXy4bIiLeGRErge8DH65RbJIkSZK0VZpW0H1H1G8zM78DfCci3gB8FjhyeJ2urq7B7dbWVlpbW8cn\nQkmSJEmqM729vfT29m7x+UWNATwE6MrMeeX9U4H1mXnOZs7pAw7OzNUVZY4BlCRJktSwJssYwBuB\nfSNiTkRsCxwLXFVZISJaIiLK268Gtq1M/iRJkiRJo1NIF9DMXBsRpwBXA1OBizJzZUScXD5+PvAe\n4PiIWAM8SylJlCRJkiRtoUK6gI4Xu4BKkiRJamSTpQuoJEmSJKnGTAAlSZIkqUGYAEqSJElSgzAB\nlCRJkqQGYQIoSZIkSQ3CBFCSJEmSGoQJoCRJkiQ1CBNASZIkSWoQJoCSJEmS1CBMACVJkiSpQZgA\nSpIkSVKDMAGUJEmSpAZhAihJkiRJDcIEUJIkSZIaxKRPANtObKNnWU/RYUiSJElS3ZtWdABjtXTO\nUvrO6wOg/cj2gqORJEmSpPo16VsAAfoO6qP70u6iw5AkSZKkurZVJIAA/ev7iw5BkiRJkupaYQlg\nRMyLiNsi4vaI+FSV4x+IiFsi4lcR8dOIeOXmrtc8pXnigpUkSZKkrUAhCWBETAXOBeYB+wPzI2K/\nYdXuBA7PzFcCZwEXbOp6LTe10DG/Y6LClSRJkqStQlEtgHOBOzLz7sxcA1wGHFNZITOvz8zHy7s/\nB15U7UJt97Sx6JRFTgAjSZIkSc+jqFlAZwP3VezfD7xuM/U/Avyg2oElX14yjmFJkiTVt4goOgRJ\nBcnMMV+jqARwxJFHxBuBDwOHVjve1dU1uN3a2kpra+sYQ5MkSapv4/EhUNLksuGPP729vfT29m75\ndYp4A4mIQ4CuzJxX3j8VWJ+Z5wyr90rgCmBeZt5R5TrpG6AkSWokEWECKDWgTf3fL5ePuGtAUWMA\nbwT2jYg5EbEtcCxwVWWFiNibUvL3wWrJnyRJkiRpdApJADNzLXAKcDVwK/DNzFwZESdHxMnlan8P\nzAK+GBErIuIX1a7V1nYGPT3LaxK3JEmSJE1mhXQBHS8RkZC0tJzOokVttLcfXnRIkiRJE8ouoFJj\nmuxdQMdVX9/ZdHcvKzoMSZIk1Ym7776bKVOmsH79+qJDUR3xudhKEkCA/v6pRYcgSZJUqJ6e5bS1\nnUFra9cWDZMZ6/kAq1ev5l3vehfTp09nzpw5XHrppZut/2//9m/ssccezJw5k4985CM899xzW3yt\nyaC3t5e99tqrpvfsWdZD24lttJ7QStuJbfQs66n5NUbzs/zLv/xLZsyYMfjV3NzMjjvuOHi8tbWV\n7bbbbvD4fvvtN+rvp97U8rkoahmIcdfcvK7oECRJkgrT07OchQuvpq/v7MGyvr7TAUY0TGas52/w\nsY99jObmZn73u9+xYsUK2tvbOfDAA9l///03qnv11VdzzjnncM0117DHHnvwrne9i8985jN87nOf\nG/W1RmNDN7pGWFOxZ1kPC89bSN9BfYNlfeeVttuPbK/ZNUbzs/zSl77El770pcH9E088kalT/9jY\nExGcd955fPjDHx7RvUeqUZ6LraIFsKXlNDo6jiw6DEmSpMIsXrx0SPIGoxsmM9bzAZ5++mmuuOIK\nzjrrLLbffnsOPfRQjjnmGC6++OKq9b/2ta/x0Y9+lP3224+ddtqJv//7v+erX/3qFl1r/fr1fOIT\nn2DXXXelpaWFnp6hLVStra2cccYZHHrooeywww7cddddXHfddRx88MHstNNOzJ07l+uvv35I/VNP\nPZXXve51zJw5k3e+85384Q9/GDx+1VVXccABBzBr1ize+MY3cttttw0emzJlCnfeeefg/gknnMCZ\nZ57JM888w9FHH82qVauYMWMGO+64Iw899NCIX98tsfiSxUMSN4C+g/rovrS7ZtcY7c9y+Lnf/va3\n+dCHPjSkfKTjYH0uNjbpE8C2tjNZtGieE8BIkqSGNjBQvWPXSIfJjPV8gP/5n/9h2rRpvOQlLxks\nO/DAA/nv//7vqvVvvfVWDjzwwMH9V77ylTz88MP84Q9/GPW1LrjgAnp6erj55pu58cYb+da3vrVR\nS843vvENLrzwQp566il22GEH2tvb+fjHP87q1av527/9W9rb24d8mL/44ov5yle+woMPPsi0adPo\n7Owc/D4XLFjA4sWLeeSRR3jrW9/K29/+dtauXVs1toggIth+++1ZsmQJe+65J08++SRPPPEEu+++\n+/O8qmMzkANVy/vX99fsGqP9WVb69re/zQtf+ELe8IY3DCk/9dRT2XXXXTnssMO49tprN3m+z8XG\nJn0CuGTJWSZ/kiSp4TU1Vf+QOdJhMmM9H+Cpp54aMlYLYMaMGTz55JObrD9z5szB/Q3nPvnkk6O+\n1uWXX87f/M3fMHv2bGbNmsVpp502pJUoIjjhhBPYb7/9mDJlCkuXLuVlL3sZH/jAB5gyZQrvf//7\nefnLX85VV101WP/4449n//33Z/vtt+ess87i8ssvZ/369Xzzm9/kbW97G29+85uZOnUqn/jEJ3j2\n2We57rrrNvnabIil1jO4NkVT1fLmKc01u8Zof5aVvva1r3H88ccPKTvnnHO46667WLVqFSeddBJv\nf/vbh7SsVfK52NikTwAlSZIEnZ1H0dJy+pCy0QyTGev5ANOnT+eJJ54YUvb4448zY8aMEdV//PHH\ngVJyMNprPfjgg0Mm0dh77703qlN5fNWqVRvV2WeffVi1alXV+nvvvTdr1qzhkUce4cEHHxxybkSw\n11578cADD1SNrUidCzppWdEypKzlphY65nfU7Bqj/VlucO+993LttddulADOnTuXHXbYgW222Ybj\njz+eQw89lB/84AdVr+FzsbGtZhIYSZKkRrahR1R395n090+luXkdHR0jHyYz1vMBXvrSl7J27Vru\nuOOOwe5+t9xyC694xSuq1j/ggAO4+eabee973ztYd7fddmPWrFlsu+22o7rWHnvswb333ju4X7m9\nQWXXv9mzZ3PFFVcMOX7PPfdw9NFHV73GvffeyzbbbMOuu+7Knnvuya9//evBY5nJfffdx+zZswHY\nfvvteeaZZwaPVyYhtZ5gZMMkLd2XdtO/vp/mKc10nNIx4slbxuMao30uNrj44os57LDDmDNnzohj\nHc7noorMnLRfpfAlSZIaR71//nn/+9+f8+fPz6effjp//OMf58yZM/PWW2+tWnfJkiW5++675623\n3pqrV6/OI444Ik899dQtutYXv/jF3H///fP+++/P1atX55ve9KaMiFy3bl1mZra2tuaFF144WP/R\nRx/NnXbaKS+55JJcs2ZNXnbZZTlr1qx89NFHMzPziCOOyBe96EV566235tNPP53vfe978wMf+EBm\nZt522225ww475H/+53/mc889l//0T/+ULS0tuWbNmszMPPTQQ/PTn/50rl27Nn/4wx/mdtttl2ee\neWZmZq5cuTK32267fPzxx8f4Sk8uo/lZbvDSl740v/KVrwwpe+yxx3LJkiX57LPP5po1a/Ib3/hG\n7rDDDnn77bdXvcbW9Fxs6v9+uXzkOdRoKtfbV72/AUqSJI23ev/8s3r16nznO9+ZO+ywQ+6zzz55\n6aWXDh675557cvr06XnfffcNlv3rv/5r7rbbbrnjjjvmhz/84XzuuedGdK3h1q5dm3/zN3+TO++8\nc774xS/O8847L6dMmTLkg/5FF1005Jyf/OQn+ZrXvCZnzpyZr33ta/OnP/3p4LHW1tY89dRTc+7c\nubnjjjvmO97xjsEkIDPzyiuvzP333z9nzpyZra2tQ5KZG2+8MQ844ICcMWNGHnfccblgwYLBD/qZ\nmR/+8Idz5513zlmzZuWDDz44mpd30hrtc3Hdddfl9OnT86mnnhpynd///vd58MEH54wZM3KnnXbK\n17/+9fmjH/1ok/fdmp6L8UoAI2s8EHU8RURO5vglSZJGKyJqPpFII3rjG9/IcccdN+5rzWlyK/K5\n2NT//XL5iPuQOgmMJEmSVIWJtqqZ7M+FCaAkSZJURa0nbNHkMNmfC7uASpIkTSJ2AZUak11AJUmS\nJEmjYgIoSZIkSQ3CheAlSZImmck+BklScUwAJUmSJhHH/0kai8K6gEbEvIi4LSJuj4hPVTn+8oi4\nPiL6I+LviohRGove3t6iQ5Cq8tlUvfLZVD3z+dTWopAEMCKmAucC84D9gfkRsd+wao8CHcA/1zg8\naVz4i0L1ymdT9cpnU/XM51Nbi6JaAOcCd2Tm3Zm5BrgMOKayQmb+PjNvBNYUEaAkSZIkbW2KSgBn\nA/dV7N9fLpMkSZIkTZBCFoKPiPcA8zLzL8r7HwRel5kdVep+BngqM/+lyjFHQUuSJElqaKNZCL6o\nWUAfAPaq2N+LUivgqIzmG5UkSZKkRldUF9AbgX0jYk5EbAscC1y1ibomeZIkSZI0DgrpAgoQEUcD\nXwCmAhdl5uci4mSAzDw/InYHbgB2BNYDTwL7Z+ZThQQsSZIkSZNcYQmgJEmSJKm2ClsIfqyebyF5\nqSgRcXdE/CoiVkTEL4qOR40tIr4cEQ9HxK8ryl4QEcsi4n8iYmlE7FRkjGpMm3g2uyLi/vL754qI\nmFdkjGpMEbFXRFwTEf8dEb+JiM5yue+dKtxmns8Rv39OyhbA8kLyvwXeQmlCmRuA+Zm5stDAJCAi\n7gJek5mri45Fiog3AE8BX8/MPy2XfR54JDM/X/4D2qzM/HSRcarxbOLZ/AzwZGb+a6HBqaGVhyHt\nnpk3R8R04JfAO4ET8b1TBdvM8/nnjPD9c7K2AD7vQvJSwZy8SHUhM38M/GFY8TuAr5W3v0bpF4dU\nU5t4NsH3TxUsMx/KzJvL208BKymtV+17pwq3mecTRvj+OVkTQBeSVz1L4EcRcWNE/EXRwUhV7JaZ\nD5e3HwZ2KzIYaZiOiLglIi6yi52KFhFzgIOAn+N7p+pMxfP5s3LRiN4/J2sCOPn6raqRHJqZBwFH\nAx8rd3OS6lKWxgH4nqp68UXgT4BXAQ8C/1JsOGpk5e513wYWZuaTlcd871TRys/ntyg9n08xivfP\nyZoAjstC8tJEyMwHy//+HriSUpdlqZ48XB5DQETsAfyu4HgkADLzd1kGXIjvnypIRGxDKfm7ODO/\nUy72vVN1oeL5/MaG53M075+TNQEczULyUs1ExPYRMaO8vQNwFPDrzZ8l1dxVwIfK2x8CvrOZulLN\nlD9Ub/AufP9UASIigIuAWzPzCxWHfO9U4Tb1fI7m/XNSzgIK1ReSLzgkiYj4E0qtfgDTgP/w2VSR\nIuJS4AhgF0pjVv4e+C5wObA3cDfw55n5WFExqjFVeTY/A7RS6r6UwF3AyRVjrqSaiIjDgOXAr/hj\nN89TgV/ge6cKtonn8zRgPiN8/5y0CaAkSZIkaXQmaxdQSZIkSdIomQBKkiRJUoMwAZQkSZKkBmEC\nKEmSJEkNwgRQkiRJkhqECaAkSZIkNQgTQElSw4uIdRGxouLrk+N47TkR4YLmkqS6MK3oACRJqgPP\nZOZBRQchSdJEswVQkqRNiIi7I+KciPhVRPw8IlrK5XMi4r8i4paI+FFE7FUu3y0iroyIm8tfh5Qv\nNTUiLoiI30TE1RHRXNg3JUlqaCaAkiTBdsO6gL6vXJ7AY5n5SuBc4Avl8m7gK5l5IPAfwOJy+WLg\nmsx8FfBq4NZy+b7AuZn5CuAx4D0T/y1JkrSxyMyiY5AkqVAR8WRmzqhSfhfwxsy8OyK2AR7MzF0i\n4vfA7pm5rly+KjN3jYjfAbMzc03FNeYASzPzpeX9TwLbZObZNfjWJEkawhZASZJGrvKvprGJOtXK\nByq21+EYfElSQUwAJUnavGMr/r2uvH0d8P7y9geA5eXt/wT+CiAipkbEjrUKUpKkkfAvkJIklccA\nVuz/MDNPK2/PiohbgH5gfrmsA/hKRPwv4HfAieXyhcAFEfERSi19fwk8zNCWQ6rsS5JUE44BlCRp\nE8pjAF+TmauLjkWSpPFgF1BJkjbNv5JKkrYqtgBKkiRJUoOwBVCSJEmSGoQJoCRJkiQ1CBNASZIk\nSWoQJoCSJEmS1CBMACVJkiSpQZgASpIaQkSsj4gXFx2HJElFMgGUJE0KEbEkIv6hSvkxEfFgRPg7\nTZKk5+EvS0nSZPFV4INVyo8DvpGZ62sbzvgxeZUk1Yq/cCRJk8V3gZ0j4g0bCiJiFtAOfD0i5kbE\n9RHxh4hYFRHdEbHNSC4cESdGxK0R8URE9EXEScOOHxMRN0fE4xFxR0S0lctfEBFfiYgHImJ1RFxZ\nLj8hIn487BqDXVAj4qsR8cWI+EFEPAW0RkR7RKwo3+PeiPjMsPMPi4jryt/fvRHxoYg4OCIeioio\nqPfuiLh5NC+sJKlxmABKkiaFzHwWuBw4vqL4z4GVmflrYC2wENgZeD3wZuCvR3j5h4H2zNwROBH4\nt4g4CCAi5gJfA/4uM2cChwN3l8+7GGgG9gdeCPzrKL6l+cBZmTkd+CnwFPDB8j3agb+KiGPKMewD\n/ABYBOwCvApYkZk3AI8CbRXXPa4cryRJG4nMLDoGSZJGJCIOBb4P7JaZz0XET4HLM3NRlbofBw7P\nzHeX99cDL8nMO0dwnyuBazJzcUScDzyVmX83rM4ewP3ACzLz8WHHTgA+kpmVrZWD94+IrwJk5gmb\nieELwPrM/NuIOBV4bWa+p0q9TwKvzMwPRsQLgPuAF2fmw8/3fUqSGo8tgJKkSSMzfwo8ArwrIlqA\ng4FLACLipRHx/fKEMI8DZ1NqDXxeEXF0RPwsIh6NiD8Ab60490VAX5XT9gJWD0/+RvqtUErUKmN4\nXURcExG/i4jHgJMrYtgL2FTi+h/A2yNie0otostN/iRJm2ICKEmabL5OqRvoB4Elmfn7cvkXgVsp\ntbLNBE5nBL/nIqIJ+DbweeCFmTmLUnfLDePq7gNeUuXU+4AXRMTMKseeBravuMfuI/i+LgG+A7wo\nM3cCvlQRw71AS7WTMvMB4Hrg3ZRek4tHcC9JUoMyAZQkTTZfB44EPsrQsW7TgSeBZyLi5cBfjfB6\n25a/HgHWR8TRwFEVxy8CToyIN0XElIiYHREvy8wHgR8C/zcidoqIbSLi8PI5twAHRMSBEdEMdA27\nZ7Cx6cAfyl1b5wILKo5dArwlIt4XEdMiYueIOHDYa/Ip4BXAFSP8viVJDcgEUJI0qWTmPZQmTdke\nuKri0CcoJU1PABcAl1Hqajl46iau9yTQSWmCmdWUJmf5bsXxGyhPDAM8BvQCe5cPHwesAW6jNJFM\nZ/mc/wH+N/Aj4LfAj6vEMjyevwb+d0Q8AZwJfLMihnspdUv9O0qTvqwAXllx7pXlmK7MzP5q36ck\nSTDBk8BExDzgC8BU4MLMPGfY8WMo/YJcT2n2to+Xx3cQEXdT+iW+DliTmXMnLFBJkia5iLgDOCkz\n/6voWCRJ9WvCEsCImErpr55vAR4AbgDmZ+bKijo7ZObT5e0/pTST237l/buA12Tm6gkJUJKkrURE\nvAf4XGa+tOhYJEn1bdoEXnsucEdm3g0QEZcBxwCDCeCG5K9sOqWWwErVxkhIkqSyiOgFXk6pO6ok\nSZs1kQngbIZOcX0/8LrhlSLincDnKC2g+9aKQwn8KCLWAedn5r9PYKySJE1KmdladAySpMljIhPA\nEfUtzczvAN+JiDcAn6U0sxvAoZn5YETsCiyLiNsy88eV50aEq9hLkiRJamiZOeKekxOZAD5AaeHa\nDfai1ApYVWb+OCJeHBEvyMzV5em1yczfR8SVlLqU/rjKeeMctjQ+urq66OrqKjoMaSM+m6pXPpuq\nZz6fqlcRoxs1N5HLQNwI7BsRcyJiW+BYhk7XTUS0RDniiHg1sG1mro6I7SNiRrl8B0rrMf16AmOV\nJEmSpK3ehLUAZubaiDgFuJrSMhAXZebKiDi5fPx84D3A8RGxBniWUpIIsDtwRTk3nAb8R2YunahY\nJUmSJKkRTGQXUDLzh8APh5WdX7H9eeDzVc67E3jVRMYmTbTW1taiQ5Cq8tlUvfLZVD3z+dTWYkIX\ngp9oEZGTOX5JkiRJGouIGNUkMBM5BlCSJEmSVEdMACVJkiSpQZgASpIkSVKDMAGUJEmSpAZhAihJ\nkiRJDcIEUJIkSZIahAmgJEmSJDUIE0BJkiRJahAmgJIkSZLUIEwAJUmSJKlBmABKkiRJUoMwAZQk\nSZKkBmECKEmSJEkNwgRQkiRJkhqECaAkSZIkNQgTQEmSJElqECaAkiRJktQgTAAlSZIkqUGYAEqS\nJElSgzABlCRJkqQGYQIoSZIkSQ3CBFCSJEmSGoQJoCRJkiQ1iGlFByBJkiRp03qW9bD4ksUM5ABN\n0UTngk7aj2wvOixNUiaAkiRJUp3qWdbDwvMW0ndQ32BZ33mlbZNAbYnIzKJj2GIRkZM5fkmSJGlz\n2k5sY+mcpRuX39PGki////buPz6uus73+PuTtCT9kZauIoVaqIygVBdkha4K2qC20zVi5a5XaVmQ\n4orug85U170KLSz1ItcHuuuahF6VC7Iu2GIfKsrtaJqgpOWH16VSwLUFYaTlR8vPQmmRhLT93D9m\nksy0k2Qmzcw5M+f1fDzy6Mx3zjn5dHLmzPmc7/l+vh0BRISwMTO5uxW7PGMAAQAAgJDq9d6C7T0H\neiocCWoFCSAAAAAQUg3WULC9sa6xwpGgVpAAAgAAACGVXJxUbHMsry12f0yJRYmAIkK1owgMAAAA\n8lB1Mjz63/f2Ne3qOdCjxrpGJZYm+Htg4HNaKorAAAAAYEChqpOxzTG1XtpK0gGERN7ndKUoAgMA\nAIDRaVvdlpf8SVL6tLTa17QHFBHCItWVUnxJXM0XNSu+JK5UVyrokCKr0Oe0WNwCCgAAgAFUnUQh\nzEcYLkN9TotBDyAAAAAGUHUShdAzHC5DfU6LQQIIAACAAVSdRCH0DOcL+nbYQp/TYnELKAAAAAaE\npeoklUjDhZ7hQWG4HTb3c7pe60talyqgAAAACBUqkYZPwb/J/TG1Lo3e3yS+JK7OWZ2Htm+Pq+P7\nHRWPx8xKqgJKDyAAAABCZbjxZlFLNsIiLD3DYVDtt8OSAAIAACBUqv0Eu1a1zGuJZMJ3sGq/HZYi\nMAAAAAiVMJ1gB13sA+FT7YWS6AEEACDCwlBoIwwxIFySi5NKr0ofMt4ssbSyJ9hhKPaBfGE4XlT7\n7bAkgAAARFQYTm7DEAPCJywn2IxFDJcwHS+q+XZYqoACABBRYahkF4YYgKE0X9SsDW/ZcEj73Mfn\nqvvfuysfUMRxvCis1CqgjAEEACCiwlBoIwwxAEMJ01hEcLwYKySAAABEVBhObsMQAzCUai/2UWs4\nXowNEkAAACIqDCe3YYgBGErLvBa1Xtqq+Pa45j4+V/Ht8UhOfB4WHC/GBmMAAQCIsJX/61pdd+v1\n2le3X+MO1GvpeZdo5fKvVDSGVFcqv9jHouqppgegsjheHKrUMYAkgAAARFQqtVHLlq1XOn3NQFss\ntkKtrXG1tHwgwMgAhFEqtVFtbZ3q7R2nhoZ9Sibnc6wIgVITQKaBAAAgotraOvOSP0lKp69Re/uV\nnNQByFPoglE6vUKSOF5UGcYAAgAQUb29ha8D9/TUVzgSAGE39AWjroAiwmiRAAIAEFENDfsKtjc2\n7q9wJADCjgtGtaOsCaCZLTCzh83sUTM7ZES5mS00swfNbLOZ3WdmZxa7LgAAODzJ5HzFYivy2mKx\n5Uok5gUUEYCw4oJR7ShbERgzq5f0iKQPS3pa0n2SFrn71pxlJrn7q9nHfylprbufXMy62XUoAgMA\nwGFIpTaqvb1LPT31amzcr0RiXmTH86S6Umpb3aZe71WDNSi5OBn56oJAv8JFo5artXVBZI8ZYRGm\nIjBzJD3m7tskycxulbRQ0kAS15/8ZU2WdKDYdQEAwOFrafkAJ2/KJH/LVi1T+rT0QFt6VeYxSSAw\nWOilvf3KnAtGJH/VqJwJ4AxJT+Y8f0rSXx+8kJl9XNLXJb1J0kdKWRcAAGAstK1uy0v+JCl9Wlrt\na9pJAIEsLhjVhnImgEXdm+nuP5P0MzN7v6SvSSpp4MHKlSsHHjc3N6u5ubmU1QEAANTrvQXbew70\nVDgSABhed3e3uru7R71+ORPApyXNzHk+U5mevILc/S4zO8HM/iK7XFHr5iaAADIYxwIApWmwhoLt\njXWNFY4EAIZ3cKfXV7/61ZLWL2cCuEnSiWY2S9IOSZ+StCh3ATOLSfqTu7uZ/ZWkI9x9l5mNuC6A\nwhjHgqGkUhvV1tap3t5xamjYp2RyPrfyAFnJxUmlV6Xzjp2x+2NKLE1UNA4u4AHD47vs8JUtAXT3\nfWa2VNJ6SfWSbnT3rWb2uezr35P0t5IuNLM+Sa8pk+gNuW65YkVt4ICQwTgWFFKoels6nSn/H8XP\nCcIlDMfv/uNj+5p29RzoUWNdoxJLExU9bnIBDxge32Vjo2zTQFQC00CgX+HSxCvU2hqP3AGh+aJm\nbXjLhkPa5z4+V93/3l35gBAK8fgV6uz8WoH2K9XRcXUAEQEZHL8HxZfE1Tmr89D27XF1fL8jgIiA\ncNv95d4AACAASURBVOG7rLBSp4Eo60TwQKW0tXXmnTxIUjp9jdrbuwKKKDiMY0Ehvb2Fb/jo6amv\ncCRAPo7fgyhEAwyP77KxQQKImsABYVBycVKxzbG8ttj9MSUWVXYcC8KloWFfwfbGxv0VjgTIx/F7\nEBfwgOHxXTY2ylkEBqgYDgiDwjCOBYcKeoxTMjlfDz3yST3Tt1sa3yv1NWj6+ClKJLgwgGBx/B4U\nlkI0QFglk/OVTq846Jbx5UokFlQ8lqC/1w8HCSBqQpgOCGHQMq+FhC9EQjFo/Yg90on3SGftGGy7\n+1jpiIsq8/uBIXD8HsQFPGB4/d+Z7e1XqqenXo2N+5VILKh44hWK7/XDQBEY1IxUaqPa27tyDgjz\nquJDiNoXhkHrFJdAmHH8BoZXzb1NtSgM3+u5Si0CQw8gakZLywc4GCKUwjDGieISCDOO38DQqr23\nqRaF4Xv9cFAEBgDKLAxjnCguAQDViUq54ROG73Upc3EgHr+i5PVIAAGgzJLJ+YrFVuS1ZcY4zatc\nDFSHDZ3+L+7m5pWKx69QKrUx6JAAFBD0Z7Xae5tqURi+1/t7hgvdijoSbgEFgDILw6B1ikuEC7d0\nhRPjrHCwMHxWw9LbhEFh+F4v1DNcLIrAAAAqJtWVUtvqNvV6rxqsQcnFyUgmoWErIIDCJ/qx2Aq1\ntsZJAiMsDJ/VwvvmcrW2Vr76JcKjuXmlNmxYmX1GERgAQAilulJatmpZ3hxn6VWZx1FLArmlK3yG\nHmd1JSfZERaGz2oYepsQPkP1DBeDBBAAUBFtq9vykj9JSp+WVvua9sglgNzSFT5hONFH+ITls0ql\nXBys0ByqxSIBBABUBFNRDEom5+uhRz6pZ/p2S+N7pb4GTR8/RYkERXmCEpYTfYRLoZPsTLGPBQFG\nBeT3DK9fX9q6JIDAGGOME1AYU1HkOGKPdOI90lk7BtvuPlY64qLAQoo6TvRRCLdfIsz6e4bNSqsE\nShEYYAwVGuMU2xxT66WtkUwCSYaRK9WV0t9/4xI9k5P0TL/7WN3w5esjt1/El8TVOavz0PbtcXV8\nvyOAiCBlim20t3flnOjP40QfQOiZUQQGCAxjnAZR8AOHeL1JevRMacsr0vgeqa9RGj8l0x4x3A4b\nToyzAhAFTAQPjCFO6gYNlwwjmtraOvXM9rXSjg5pe7e0o0PPbF+r9vauoEOrOG6HBQAEhR5AYAxx\nUjcoLMkwEzuHB1UWByUXJ5Velc6/Xfz+mBJLKQITdRyzAJQbCSAwhjipGxSGZLjQ5Lnp9ApJqvgJ\nFeMhqbKYq/9v376mXT0HetRY16jE0kTk9gnkC9MxC0DtIgEExhAndYPCkAyHZWJnxkNmUGUxX8u8\nlkj9/TGysByzANQ2EkCMCXo3BnFSlxGGZDgstxxSHCiDcurA8MJyzAJQ20gAcdjo3cBQgk6Gw3LL\nYVjGQ4YBVRaBoYXlmAWgtlEFFIeNao8Iq2RyvmKxFXltmVsO51U0jjCMhwQQfmE5ZgGobfQA4rDR\nu4GwCssth2EYDwkg/MJyzJKoRgrUMhJAHLaw9G4wDhGFhOGWwzCMhwRQHcJwzKIaKVDbzN2DjmHU\nzMyrOf5aUWgMYOz+mFqXtlbsBLdgDJtjar20cjEAAFAL4vEr1Nn5tQLtV6qj4+qKxUEvJFAcM5O7\nW7HL0wOIwxaG3g2qLAIAMDbCUI2UXkigfEgAMSaCrvbIOMR8XDUFAIxWGKqRMiciUD4kgKgJYRmH\nGAZcNQUAHI5kcr7S6RV53yOZaqQLKhZDGHohgVpFAoiaQJXFQVw1BQAcjjBUIw1DLyRQq0gAURPC\nMA4xLLhqCgA4XEFXIw1DLyRQq0gAUTOCHocYFlw1BQBUuzD0QgK1imkggBpTaAxgLLZcra18cQIA\nANSaUqeBIAEEalAqtVHt7V05V03nkfwBAADUIBJAIGBMwTCI9wIAAKC8mAgeCBBTMAzivUCYcXEC\nABBVI/YAmtnHJK1z9wOVCal49AAibOLxK9TZ+bUC7Veqo+PqACIKDu8FwqrwONkVam2NkwQCAKpO\nqT2AdUUs8ylJj5nZN8zs7aMPDah9TMEwiPcCYTX0XJldAUUEAEDljJgAuvv5kk6T9CdJ/25mvzGz\nS8ysqezRAVWGKRgG8V4grLg4AQCIsmJ6AOXuuyX9WNKPJB0r6VxJm80sWcbYgKqTTM5XLLYiry0z\nce28gCIKTljei1RXSvElcTVf1Kz4krhSXamK/n6EDxcnAABRNmIRGDNbKOkiSSdK+g9JZ7j7c2Y2\nUdIWSW1ljRCoIkxcOygM70WqK6Vlq5YpfVp6oC29KvO4ZV5LxeJAuCST8/XQI5/UM327pfG9Ul+D\npo+fokQiEXRoAACUXTFFYH4g6UZ331jgtQ+7+x3lCm4kFIEBMJz4krg6Z3Ue2r49ro7vdwQQEcIg\n1ZXS33/jEj1z1o6Btul3H6sbvnw9FwYAAFWnHEVgvirpvpxfMMHMZklSkMkfAIyk13sLtvcc6Klw\nJAiTttVtecmfJD1z1g61r2kPKCIAACqnmARwraTcgREHlBkPCACh1mANBdsb6xorHAnChAsDAIAo\nKyYBHOfur/c/cfdeSePLFxIAjI3k4qRim2N5bbH7Y0osYqxXlHFhAAAQZSMWgZH0gpktdPefSwNF\nYV4ob1gAcPj6x3O1r2lXz4EeNdY1KrE0wTiviEsuTiq9Kp1XHCh2f0yJpVwYAADUvmKKwLxV0g+V\nmf5Bkp6SdIG7P1bm2EZEERgAwGikulL5FwYWcWEAAFCdSi0CM2ICmLPhJknu7ntHG9xYIwEEAAAA\nEGWlJoDF3AIqM/uopNmSGs0y23b3/zmqCAEAAAAAgRixCIyZfU/SJyUlJVn28fFljgsAAAAAMMaK\nGQP4e3f/SzN7yN1PMbPJkjrc/azKhDhsbNwCCgAAACCyynEL6GvZf/9sZjMkvShp+miCqzWprpTa\nVrep13vVYA1KLk5SRAAAAABAaBWTAP5fM5sm6ZuSfpdt+z/lC6k6pLpSWrZqWV4Z8fSqzGOSQAAA\nAABhNOwtoGZWJ+m97n5P9nmjpEZ3f7mojZstkPRtSfWSbnD3aw96/XxJX1ZmbOEeSf/g7g9lX9sm\n6RVJ+yX1ufucAtsP7BbQ+JK4Omd1Htq+Pa6O73cEEBEAAACAqBnTW0Dd/YCZrZL0ruzzHkk9RQZS\nL+k6SR+W9LSk+8zsdnffmrPYnyR9wN13Z5PF6yW9p//XS2p2913F/mcqqdd7C7b3HCjq7QEAAACA\nihuxCqikO8zsE9Y//0Px5kh6zN23uXufpFslLcxdwN1/4+67s09/K+nNB22j1N9ZMQ3WULC9sa6x\nwpEAAAAAQHGKSQA/L2mtpNfNbE/255Ui1psh6cmc509l24byGUm/yHnuyiSfm8zss0X8vopKLk4q\ntjmW1xa7P6bEokRAEQEAAADA8EYsAuPuk0e57aIH55nZ2ZIulnRmTvOZ7r7TzI6S1GVmD7v7XQev\nu3LlyoHHzc3Nam5uHmW4pekv9NK+pl09B3rUWNeoxNIEBWAAAAAAlE13d7e6u7tHvX4x8wB+oFC7\nu28cYb33SFrp7guyzy+XdKBAIZhTJP1U0gJ3f2yIbV0laa+7/+tB7cwDCAAAACCyyjEP4Jc12JvX\nqMzYvt9J+uAI622SdKKZzZK0Q9KnJC06KNjjlEn+/i43+TOziZLq3X2PmU2SNF/SV4uINXKYixAA\nAABAsYq5BfSjuc/NbKak1iLW22dmSyWtV2YaiBvdfauZfS77+vck/bOkaZK+k60x0z/dw3RJP822\njZP0Q3c/dM6FiGMuQgAAAAClGPEW0ENWyGRlW9z95PKEVFIskb4FlLkIAQAAgGgb81tAzaw952md\nMnMC/m4UsWGMMRchAAAAgFIUMwbwdxocA7hP0mp3v6d8IaFYzEUIAAAAoBTFJIA/lvSau++XJDOr\nN7OJ7v7n8oYWfqnURrW1daq3d5waGvYpmZyvlpaCRVPLIrk4qfSqdN4YwNj9MSWWMhchAAAAgEMV\nkwDeIenDkvZmn09UprDL+8oVVDVIpTZq2bL1SqevGWhLp1dIUsWSQOYiBAAAAFCKYuYBfMDd3zVS\nWxCCLAITj1+hzs6vFWi/Uh0dVwcQUbCC7g0FAAAAoqgc8wC+ambvdvffZX/B6ZJeG22AtaK3t/Bb\n19NTX+FIgheG3lAAAAAAI6srYpkvSFprZneb2d2SfiQp8oPMGhr2FWxvbNxf4UiC19bWmZf8SVI6\nfY3a27sCiggAAABAIcVMBH+fmZ0s6W3Zpkfc/fXyhhV+yeR8pdMr8hKfWGy5EokFAUYVDHpDAQAA\ngOpQzDyASyX90N1/n30+zcz+3t3/d9mjC7H+Wxvb269UT0+9Ghv3K5FYEMlbHukNBQAAAKpDMUVg\nHnT3Uw9qi3wRGAwqNAYwFluu1tZoJsQAAABApZSjCEydmdW5+4HsL6iXNH60AWJshaH6Jr2hAAAA\nQHUopgfwXyQdJ+l7kkzS5yQ94e5fKn94w4t6D2DhnrcVam2Nk3wBAAAAEVBqD2AxVUC/IulOSf+g\nTPL3kKQJowsPY4nqmwAAAABKMWIC6O77Jf1W0jZJcyR9SNLW8oaFYlB9EwAAAEAphhwDaGZvk7RI\n0nmSXlRm/j9z9+bKhIaRUH0TAAAAQCmG6wHcKulsSee4+5nu3iaJzCJEksn5isVW5LVl5iKcF1BE\nAAAAAMJsuCqg/02ZHsA7zeyXyvYAViQqFIXqmwAAAABKUUwV0MmSFiqTDJ4t6T8k3ebuneUPb3hR\nrwIKAAAAINpKrQI6YgJ40Mb/QtInJJ3n7h8cRXxjigQQAAAAQJSVNQEMGxJAAAAAAFFWjnkAAQAA\nAAA1gAQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAA\nACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKCBBAAAAAA\nIoIEEAAAAAAiggQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKCBBAAAAAAIoIEEAAAAAAi\nggQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKCBBAAAAAAIoIEEAAAAAAiggQQAAAAACKC\nBBAAAAAAIoIEEAAAAAAiggQQAAAAACKirAmgmS0ws4fN7FEz+0qB1883swfN7CEzu8fMTil2XQAA\nAABAaczdy7Nhs3pJj0j6sKSnJd0naZG7b81Z5r2Strj7bjNbIGmlu7+nmHWz63u54gcAAACAsDMz\nubsVu3w5ewDnSHrM3be5e5+kWyUtzF3A3X/j7ruzT38r6c3FrgsAAAAAKE05E8AZkp7Mef5Utm0o\nn5H0i1GuCwAAAAAYwbgybrvoezPN7GxJF0s6s9R1V65cOfC4ublZzc3Nxa4KAAAAAFWlu7tb3d3d\no16/nGMA36PMmL4F2eeXSzrg7tcetNwpkn4qaYG7P1biuowBBAAAABBZYRoDuEnSiWY2y8yOkPQp\nSbfnLmBmxymT/P1df/JX7LoAAAAAgNKU7RZQd99nZkslrZdUL+lGd99qZp/Lvv49Sf8saZqk75iZ\nJPW5+5yh1i1XrAAAAAAQBWW7BbQSuAUUAAAAQJSF6RZQAAAAAECIkAACAAAAQESQAAIAAABARJAA\nAgAAAEBEkAACAAAAQESQAAIAAABARJAAAgAAAEBEkAACAAAAQESQAAIAAABARJAAAgAAAEBEkAAC\nAAAAQESQAAIAAABARJAAAgAAAEBEkAACAAAAQESQAAIAAABARJAAAgAAAEBEkAACAAAAQESQAAIA\nAABARJAAAgAAAEBEkAACAAAAQESQAAIAAABARFR9AhhfEleqKxV0GAAAAAAQeuOCDuBwdc7qVHpV\nWpLUMq8l4GgAAAAAILyqvgdQktKnpdW+pj3oMAAAAAAg1GoiAZSkngM9QYcAAAAAAKFWMwlgY11j\n0CEAAAAAQKjVRAIYuz+mxKJE0GEAAAAAQKhVfRGY+Pa4EksTFIABAAAAgBGYuwcdw6iZmVdz/AAA\nAKUys6BDABCQQrmPmcndiz4wVH0PIAAAQNRwARyInrG6+FMTYwABAAAAACMjAQQAAACAiCABBAAA\nAICIIAEEAAAAgIggAQQAAEDN2bZtm+rq6nTgwIGgQ0GIsF+QAAIAAGAM7dq1S+eee64mT56sWbNm\nac2aNcMu/2//9m865phjNHXqVH3mM5/R66+/PuptVYPu7m7NnDkz6DAqrpS/5ec//3k1NTUN/DQ2\nNmrKlCkDrzc3N2vChAkDr5988smV+C+UVSX3C6aBAAAAqBGp1Ea1tXWqt3ecGhr2KZmcr5aWD1Rs\nfUm69NJL1djYqOeee06bN29WS0uLTj31VM2ePfuQZdevX69rr71Wd955p4455hide+65uuqqq/T1\nr3+95G2Von8ajajMqZjqSqltdZt6vVcN1qDk4qRa5rVUdBul/C2/+93v6rvf/e7A8yVLlqi+vn7g\nuZlp1apVuvjii0v6P4wkMvuFu1ftTyZ8AACA6Bjq/Gfdug0eiy13yQd+YrHlvm7dhqK2e7jru7vv\n3bvXjzjiCH/00UcH2i688EK/7LLLCi6/aNEiX7FixcDzX//61z59+vRRbWv//v3+pS99yd/4xjf6\nCSec4Nddd52bme/fv9/d3efOnesrVqzw973vfT5hwgRPp9N+zz33+Omnn+5Tp071M844w++9996B\n7c2dO9cvu+wynzNnjk+ZMsUXLlzou3btGnj95z//uc+ePduPPPJIb25u9q1btw68ZmaeTqcHnn/6\n05/2K664wl999VVvbGz0uro6nzx5sjc1NfnOnTuLem9Ha13nOo8tjLlWauAntjDm6zrXVWwbpf4t\nD163qanJN27cONDW3NzsN9xwQ1G/u5b2i6E++9n2onMobgEFAACoAW1tnUqnr8lrS6evUXt7V0XW\nl6Q//vGPGjdunN761rcOtJ166qn6wx/+UHD5LVu26NRTTx14fsopp+jZZ5/VSy+9VPK2rr/+eqVS\nKT3wwAPatGmTfvzjHx/Sk3PLLbfohhtu0N69ezVp0iS1tLToC1/4gnbt2qV//Md/VEtLi1566aWB\n5W+++WbddNNN2rlzp8aNG6dkMjnw/1y8eLHa2tr0wgsv6CMf+YjOOecc7du3r2BsZiYz08SJE9XR\n0aFjjz1We/bs0SuvvKLp06eP8K4enrbVbUqfls5rS5+WVvua9opto9S/Za6f/OQnetOb3qT3v//9\nee2XX365jjrqKJ111lnasGHDkOuzXxyKBBAAAKAG9PYWHtnT01NfsH2s15ekvXv35o3VkqSmpibt\n2bNnyOWnTp068Lx/3T179pS8rbVr1+qLX/yiZsyYoWnTpmn58uUDt/RJmZPtiy66SCeffLLq6urU\n2dmpt73tbTr//PNVV1en8847T29/+9t1++23Dyx/4YUXavbs2Zo4caKuvvpqrV27VgcOHNCPfvQj\nffSjH9WHPvQh1dfX65/+6Z/02muv6d577x3yvemPJTemSuj13oLtPQd6KraNUv+WuX7wgx/owgsv\nzGu79tpr9fjjj2vHjh265JJLdM455+hPf/pTwfXZLw5FAggAAFADGhoK9zI0Nu6vyPqSNHnyZL3y\nyit5bbt371ZTU1NRy+/evVtSJjkodVs7d+7MK6Jx3HHHHbJM7us7duw4ZJnjjz9eO3bsKLj8cccd\np76+Pr3wwgvauXNn3rpmppkzZ+rpp58uGFuQGqyhYHtjXWPFtlHq37LfE088oQ0bNhySAM6ZM0eT\nJk3S+PHjdeGFF+rMM8/UL37xi4LbYL84FAkgAABADUgm5ysWW5HXFostVyIxryLrS9JJJ52kffv2\n6bHHHhtoe/DBB/XOd76z4PLveMc79MADD+Qte/TRR2vatGklb+uYY47RE088MfA893G/3Fv/ZsyY\noe3bt+e9vn37ds2YMaPgNp544gmNHz9eRx11lI499ti8dd1dTz755MC6EydO1J///OeB13fu3Dnw\nuytdYCS5OKnY5lheW+z+mBKLEhXbRql/y34333yzzjrrLM2aNavoWA/GflFAKQMGw/YjisAAAICI\nGe78Z926DR6PX+Fz517l8fgVJRVwGYv13d3PO+88X7Rokb/66qt+1113+dSpU33Lli0Fl+3o6PDp\n06f7li1bfNeuXT537ly//PLLR7Wt73znOz579mx/6qmnfNeuXf7BD34wr9jHwYVDXnzxRT/yyCN9\n9erV3tfX57feeqtPmzbNX3zxRXfPFPt485vf7Fu2bPFXX33VP/GJT/j555/v7u4PP/ywT5o0yX/1\nq1/566+/7t/85jc9Fot5X1+fu7ufeeaZftlll/m+ffv8l7/8pU+YMMGvvPJKd3ffunWrT5gwwXfv\n3l3yezta6zrXeXxJ3Od+eq7Hl8RLKgAzVtso5W/Z76STTvKbbropr+3ll1/2jo4Of+2117yvr89v\nueUWnzRpUl6BmVy1tF8M9dlXiUVgAk/iDudHks+fv2JUBycAAIBqFPYL4Lt27fKPf/zjPmnSJD/+\n+ON9zZo1A69t377dJ0+e7E8++eRA27e+9S0/+uijfcqUKX7xxRf766+/XtS2DrZv3z7/4he/6G94\nwxv8hBNO8FWrVnldXV3eif6NN96Yt87dd9/t7373u33q1Kl++umn+z333DPwWnNzs19++eUD1R4/\n9rGPDSQB7u633Xabz54926dOnerNzc15ycymTZv8He94hzc1NfkFF1zgixcvHjjRd3e/+OKL/Q1v\neINPmzat7FVAw6LU/eLee+/1yZMn+969e/O28/zzz/sZZ5zhTU1NfuSRR/p73/tev+OOO4b8vbW0\nX4xVAmhe4YGoY8nMXHLFYivU2hoveZ4aAACAamNmFS8kEkVnn322LrjggjGfaw7VLcj9YqjPfra9\n6HtIa2IMYKkligEAAICRkGijkGrfL2oiAZRKK1EMAAAAjKTSBVtQHap9vyg84UsVKqVEMQAAADCc\nO++8M+gQEEK1sF/URA9gqSWKAQAAACCKqr4HMB6/UonEAgrAAAAAAMAIqr4KaDXHDwAAUCqqgALR\nNFZVQKu+BxAAACBqqr0IBYDgkAACAABUEXr/AByOshaBMbMFZvawmT1qZl8p8Prbzew3ZtZjZl86\n6LVtZvaQmW02s/8sZ5xAOXR3dwcdAlAQ+ybCin0TYcb+iVpRtgTQzOolXSdpgaTZkhaZ2ckHLfai\npISkfymwCZfU7O6nufuccsUJlAtfFAgr9k2EFfsmwoz9E7WinD2AcyQ95u7b3L1P0q2SFuYu4O7P\nu/smSX1DbIMb3AEAAABgjJQzAZwh6cmc509l24rlku4ws01m9tkxjQwAAAAAIqhs00CY2d9KWuDu\nn80+/ztJf+3uiQLLXiVpr7v/a07bMe6+08yOktQlKeHudx20HqOgAQAAAERaWKaBeFrSzJznM5Xp\nBSyKu+/M/vu8md2mzC2ldx20DLeIAgAAAECRynkL6CZJJ5rZLDM7QtKnJN0+xLJ5iZyZTTSzpuzj\nSZLmS/p9GWMFAAAAgJpXth5Ad99nZkslrZdUL+lGd99qZp/Lvv49M5su6T5JUyQdMLNlylQMfZOk\nn2YnOR0n6Yfu3lmuWAEAAAAgCso2BhAAAAAAEC5lnQi+nEaaZB4IipltM7OHzGyzmf1n0PEg2szs\n+2b2rJn9PqftL8ysy8z+aGadZnZkkDEimobYN1ea2VPZ4+dmM1sQZIyIJjObaWZ3mtkfzOy/zCyZ\nbefYicANs38Wffysyh7A7CTzj0j6sDLFZu6TtMjdtwYaGCDJzB6X9G533xV0LICZvV/SXkn/4e5/\nmW37hqQX3P0b2Qto09z9siDjRPQMsW9eJWmPu38r0OAQadkhStPd/QEzmyzpd5I+LmmJOHYiYMPs\nn59UkcfPau0BHHGSeSBgVKhFKGSnz3npoOaPSfpB9vEPlPniACpqiH1T4viJgLn7M+7+QPbxXklb\nlZnLmmMnAjfM/ikVefys1gTwcCeZB8rJJd1hZpvM7LNBBwMUcLS7P5t9/Kyko4MMBjhIwsweNLMb\nucUOQTOzWZJOk/RbcexEyOTsn/8v21TU8bNaE8Dqu28VUXKmu58m6W8kXZq9zQkIJc+MA+CYirD4\njqS3SHqXpJ2S/jXYcBBl2dvrfiJpmbvvyX2NYyeClt0/f6zM/rlXJRw/qzUBPKxJ5oFycved2X+f\nl3SbMrcsA2HybHYMgczsGEnPBRwPIEly9+c8S9IN4viJgJjZeGWSv5vd/WfZZo6dCIWc/fOW/v2z\nlONntSaApUwyD1SMmU00s6bs40mS5kv6/fBrARV3u6RPZx9/WtLPhlkWqJjsSXW/c8XxEwGwzETU\nN0ra4u7fznmJYycCN9T+WcrxsyqrgEqSmf2NpG9rcJL5rwccEiAze4syvX6SNE7SD9k3ESQzWyNp\nrqQ3KjNm5Z8l/VzSWknHSdom6ZPu/nJQMSKaCuybV0lqVub2JZf0uKTP5Yy5AirCzM6StFHSQxq8\nzfNySf8pjp0I2BD753JJi1Tk8bNqE0AAAAAAQGmq9RZQAAAAAECJSAABAAAAICJIAAEAAAAgIkgA\nAQAAACAiSAABAAAAICJIAAEAAAAgIkgAAQCRZ2b7zWxzzs+Xx3Dbs8yMCc0BAKEwLugAAAAIgT+7\n+2lBBwEAQLnRAwgAwBDMbJuZXWtmD5nZb80slm2fZWa/NrMHzewOM5uZbT/azG4zsweyP+/Jbqre\nzK43s/8ys/Vm1hjYfwoAEGkkgAAASBMOugX0v2fbXdLL7n6KpOskfTvb3i7pJnc/VdIPJbVl29sk\n3enu75L0V5K2ZNtPlHSdu79T0suS/rb8/yUAAA5l7h50DAAABMrM9rh7U4H2xyWd7e7bzGy8berx\nFAAAAQZJREFUpJ3u/kYze17SdHffn23f4e5Hmdlzkma4e1/ONmZJ6nT3k7LPvyxpvLtfU4H/GgAA\neegBBACgeLlXTW2IZQq19+Y83i/G4AMAAkICCADA8D6V8++92cf3Sjov+/h8SRuzj38l6R8kyczq\nzWxKpYIEAKAYXIEEACA7BjDn+S/dfXn28TQze1BSj6RF2baEpJvM7H9Iek7Skmz7MknXm9lnlOnp\n+7ykZ5Xfc6gCzwEAqAjGAAIAMITsGMB3u/uuoGMBAGAscAsoAABD4yopAKCm0AMIAAAAABFBDyAA\nAAAARAQJIAAAAABEBAkgAAAAAEQECSAAAAAARAQJIAAAAABExP8HLHdl63oyuu8AAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa0c179b450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question\n",
    "Explain what you see in this experiment. What does it suggest about dropout?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Answer\n",
    "The model with dropout is better at first "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
